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Abstract 

The  vector  moving  average  process  is  a stationary  stochastic  process 
{yfc}  satisfying  y^  = where  the  unobservable  process 

consists  of  independently  identically  distributed  random  variables.  The 
matrix  parameters  E(s)  = £3^^,  s « 0,  1,  q are  estimated  from 

the  observations  y^  The  likelihood  function  is  derived  under 

normality  and  to  solve  the  maximum  likelihood  equations  the  Newton-Raphson 
and  Scoring  methods  are  used.  The  estimation  problem  is  considered  in  the 
time  and  frequency  domains.  Asymptotic  efficiency  of  the  estimates  is 
established. 

Key  words:  Maximum  likelihood  estimation,  vector  moving  average  models, 
Newton-Raphson  and  Scoring  iterative  procedures.  Time  and  Frequency 


Domains. 
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Average  Models  in  the  Time  and  Frequency  Domains 

by 

F.  Ahrabi 

Stanford  University 

1.  Introduction 

There  have  been  a number  of  papers  dealing  with  the  estimation  of 
the  vector  autoregressive  moving  average  (VARMA)  models.  Hannan  (1969, 

1970)  has  considered  the  problem  in  the  pure  moving  average  case  in  the 
frequency  domain.  Nicholls  (1976)  has  extended  this  to  the  case  of 
VARMA  models  which  include  exogenous  variables.  Reinsel  (1976)  has 
considered  the  problem  in  the  time  domain.  There  have  been  other  papers 
in  this  area,  among  them  Akaike  (1973),  Tunnicliffe  Wilson  (1973), 

Kashyap  (1970),  Whittle  (1963),  and  Osborn  (1977),  In  all  these  papers 
the  parameters  of  interest  are  the  matrix  coefficients  of  the  vectors  of 
observable  and  unobservable  random  variables  and  the  common  variance 
covariance  matrix  of  the  unobservable  random  variables. 

This  paper  is  concerned  with  the  estimation  of  vector  moving  average 
models,  but  following  Anderson  (1975),  Parzen  (1971),  and  Clevenson 
(1970)  we  take  as  cur  parameters  the  autocovariance  matrices  of  the 
observable  random  variables. 

There  is  one  paper  by  Newton  (1975)  which  is  primarily  concerned 
with  the  usual  parameterization  of  the  VARMA  models.  However  he  derives 
estimates  for  the  autocovariance  matrices  in  the  pure  moving  average 
case.  His  method  is  to  regress  the  elements  of  the  sample  spectral 

Acknowledgement . I would  like  to  thank  my  adviser,  Professor  T.  W.  Anderson, 
under  whose  supervision  and  guidance  this  work  was  completed. 
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density,  evaluated  at  a number  of  equidistant  points,  on  certain  trigo- 
nometric functions  using  the  method  of  weighted  least  squares.  The 
estimate  seems  to  be  different  from  the  estimates  derived  in  this  paper. 

The  methods  used  in  this  paper  are  the  Newton-Raphson  and  Scoring 
Methods,  applied  to  the  maximum  likelihood  equations  in  the  time  and 
frequency  domains.  The  likelihood  function  is  derived  under  the 
assumption  of  normality  of  the  data. 

To  summarize.  Section  2 describes  the  model  and  the  parameters  to 
be  estimated.  Sections  3 and  4 deal  with  the  estimation  problem  in  the 
time  and  frequency  domains  respectively.  Finally,  in  Section  5 we 
derive  the  limiting  average  information  matrix  and  show  that  the  esti- 
mates proposed  are  consistent  and  have  the  desired  limiting  multivariate 
normal  distribution,  i.e.  they  are  asymptotically  efficient. 
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2 , The  Model 


We  have  the  observations  y , y , , y , where 


(2.1) 


y = e.  + A.  e . + a e 

~t  ~t  ~1  ~t-l  ~q  ~t-q 

pXl  pXp  pXl 


Assumption  1:  The  £ ' s are  i ■ i . d , with  mean  zero  and  unknown 
covariance  matrix  V . 

Assumption  2;  The  roots  of  the  determinants!  equation 


(2.2) 


A 

1 1 + A.  z + A„z  + ...  + A zC' 

— ~_L  1 


lie  outside  the  unit  circle. 

Note:  Assumption  2 enables  us  to  recover  the  coefficients  A , A , 

~1  ~2 

•••»  A from  the  autocovariance  matrices  uniquely.  The  latter  are  the 
parameters  of  interest  xvhich  are 


z(0)  - e(yty[)  , 


1(S)  “ £(Zt?t+s)5  s=1’  2’ 


For  ease  of  differentiation  of  the  log  likelihood  function  we  should  vec- 
torize these  matrices  where  we  use  the  notation 


where 


Vec  A = 


5 


^ (s,  y o e b 5 3 ) 

and  the  a±’s  are  column  vectors.  But  we  notice  that  is  symmetric 

and  hence  should  be  treated  separately.  So  we  vectorize  the  diagonal  and 

(0) 

subdiagonal  elements  of  £ separately.  So  the  parameters  are 
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e^1)  - dg(z(0))  = 


I4V\ 


\C/ 


0q2)  = Vec  z(0) 


where  the  operator  Vec  vectorizes  any  matrix  that  it  is  applied  to, 
ignoring  the  diagonal  and  upper  diagonal  elements  of  that  matrix,  e.g. 


1 2 9\ 

Vec  | 4 3 5 

.0  6 7/ 


4 

0 i > 
.6 


and  finally 


0 = Vec  s=l,  2,  q . 


Now  we  can  put  all  these  vectors  in  a single  £qp2  + x 1 vector 


i.e. 


where 


" <®0’  ~0i’  ••-?q) 


n.  _ /c’d)  n'  (2)  s 

®o  ■ (?0  ’ ®o  } 


Remarks . 

(i)  We  can  find  a matrix  B such  that 

~ 2 

pxp 


(2.3) 


dg(  A ) = B Vec  A 
PXP 
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B is  obtained  from  the  p xp  identity  matrix  by  deleting  all  the  rows 

2 

except  1st,  p+2nd,  2p+3rd,  p th,  i.e. 


(2.4) 


where 


1 9 ^^1*  e2*  • • • * ® 9) 


~ 2 
P 


(ii)  In  a similar  manner  we  can  find  a — ^ ^ x 


p matrix 


such  that 


(2.5)  Vec  A = C Vec  A . 

PXP 

C is  obtained  from  I ^ by  deleting  the  following  rows 

~P 

1,  p+1,  2p+l,  ...,  (p-l)p+l 

P+2,  2p+2 (p-l)p+2 

2 P+3,  ...,  (p-l)p+3 

(p-l)p+p  . 

We  shall  find  it  convenient  to  introduce  another  vector  9 where 

(2.6)  0f  = (Vec’  E(0),  Vec’  E(1),  ...»  Vec’  Z(q)) 

•V  ^ 

^0’  ~1!  ' 
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CJ  ! 


3.  Estimation  in  the  Time  Domain 


3.1.  Introduction 

We  are  going  to  use  maximum  likelihood  estimation  and  proceed  as 
if  the  e ' s in  Section  2 are  normally  distributed  and  we  shall  show 
later  that  the  resulting  estimates  have  the  same  limiting  covariance 
matrix  irrespective  of  £t's  being  normal. 

Let 


Then 

y ~ n(o,Z)  , 

where 


6 


Now  using  Kronecker  products  and  the 


TxT  matrix 


similar  to  the  one  introduced  by  Anderson  (1975)  we  can  write  £ as 

(3.1.1)  £ = IT  ® £(0)  + (L  © £(1)  + L*  @ £,(1))  + ...  + 

(Lq  © £^  + b'q  © £*^) 

The  log  likelihood  of  y is 

T 

log  £(y)  = - - log  2n  -%  log | £ | - ig  y'£-1y  . 


The  maximum  likelihood  estimates  are  a set  of  the  roots  of  the  equation 

3 log  £(y) 


So  we  proceed  to  find  the  first  derivative  of  the  log  likelihood,  and 
in  doing  so  we  use  the  fact  that 


'3  log  £(y )' 

30 


= 0 


which  means 


(3.1.2) 


3 log  £(y) 
30 


y + h& 


3.2.  The  First  Derivative  of  log  £(y) 

As  noted  above  we  only  need  to  find  ~ y'£_1y. 

venient  to  find  ~ y’£_1y  and  then,  noting  that 
30  ~ ~ ~ 


It  is  more  con- 


7 


(3.2.1) 


3 


3a<°> 

13 


y'Z  y = 


-l  3Z  -1 

-y'5  —m  1 y 

do K ~ ~ 

13 

-y’S"1  (IT  © E..)E-1y 

~ ~ -1  ~13  ~ ~ 

-y'^"1  (IT  © E . ) E_1y 


A_  ...v-i 


3a 


(0) 

ji 


y'z  y , 


where  is  a matrix  with  1 for  the  ijth  element  and  0's  else- 

where, we  get 


(3.2.2) 


-rfoT  y’z  xy  = 2 —7m  y'z  \ > 

3ct;u'  ~ ~ ~ 8a(0)  ~ ~ ~ 

i;  ij 


where  — (0)  indicates  that  we  take  the  symmetry  of  into  account. 

3a. . 

13 

The  end  result  is  that 


(3.2.3) 


?0  y'5  y _ 9 — y'z  Xy  , 


36 


where  G is  a qp  + — x (q+l)p2  matrix  which  can  be  written  as 


(3.2.4) 


P(P-1) 

2 

2 

qp 


and 


2l  = (?>  °>>  ?2  = (2£’  0>*  G3  = (0,  I)  , 

with  B and  C as  in  Section  2. 
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So  we  proceed  to  find 


80 


y!£  5'y,  using  the  fact 


dZ 


= -z  — — y 

8x  i 


We  shall  also  find  it  convenient  to  express  y’E  1y  differently  using 
the  identity 

Vec (ABC)  = (C'  0 A)  Vec  B , 

[see  Mine  and  Marcus  (1964)]  which  enables  us  to  write 


y'E  1y  = Vec(y'E  1y)  = (y'  © y')  Vec  Z 1 


So  we  only  have  to  differentiate  Vec  Z 1,  but 
-1 


8 Vec  Z 
8x 


- = -Vec 


/ -1  -1\  -1  l 0Z 

(*  * 5 ) " © r > Vec  8§ 


Now 


8£ 


8a: 

ij 


(0)  It  © 5ij  ’ j=1»  , p , 


So 


8E 

r— v =LS@E..+LS©E  , i.  i=l  d 

8afs  ~ ^ ~ P 

ij 


8 Vec  if1 

~(0)  = _(5  ©5  > Vec(jT  © Eij)>  i»  j=1>  •••.  p 

ij 


- ~(Z  ® Z pf?^  say. 
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which  yields 


(3.2.5) 


3 Vec  E 


30’ 


-1 


= -(E  1 © E_1)E0  , 


where 


(3.2.6) 


F _ fn(0)  AO)  (0) 

~o  » £21  ’ •••»  5pP  } 


(s) 

Similarly,  differentiating  w.r.t.  a;.  we  get 


3 Vec  E 

3afs) 
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-1 


= -(E  1 © E 1)  Vec(LS  © E..  + L 
~ ~ ~ ~ij  ~ 


= -c?"1  ® r1)  pi f > i.  3=1, 


which  yields 


(3.2.7) 


3 Vec  E 


30' 

~s 


-1 


■(E-1  © E"1)  E , s = 1, 

~ ~ ~ s 


where 


E = 


(P(S)  P(S)  P(S)) 

kPll  ’ P21  ’ *•*»  Ppp  ' 


From  (3.2.5)  and  (3.2.7)  we  get 


(3.2.8) 


3 Vec  E 


30' 


-1 


= -(E  1 © E 1)  E , 


where 


(3.2.9) 


E (Eq,  ^1’  ' ' ’ ’ E ) > 


which  in  turn  yields 


(3.2.10) 


Sy'Z^y 


" = -Cy'2  1 0 y'Z-1)  E 


3 log  £(y) 

Now  to  complete  the  computation  of  — — --  ~ we  have  to  take  the 

30 

expectation  of  (3.2.10).  We  note  that  for  any  two  vectors  u,  v (of 
the  same  dimension)  we  have 


This  means 


Vec(uv')  = v © - • 


(y’E  1 © y'£-1)  = Vec  ’ (Z  Xyy ' Z -1-) 


e(y'£  1 © y'Z  x)  = Vec’  efS^yy'S  X) 


, j v -1- 


= Vec’  Z * . 


Therefore 


3 log  £(y) 


= %(y'2  1 © y ' Z~ 1 - Vec*  Z X)  E , 


3 log  £(y) 


= % EU^y  © Z-1y  - Vec  Z"1) 


Finally 


3 log  £(y) 

(3.2.11)  — =1-  = k GE * (Z  v (x)  Z_]h 


“ ^ GEf  (E  y (x)  £ y - Vec  £ ) 


Note:  We  have  from  above 


3 log  £(y) 


= * ?i(!  ? © 5 \ - Vec  Z-1) 
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Now 


Ejfif1,  ® r'y)  - 


V^T1*  2r1z 

x ip 


til 

where  Vec  A is  the  r column  of  E.  and  A.  is  TpxTp. 
~ir  ~i  ~ir 


So  we 


get 


E!(E-1y  © = (I  9 ® y’S  X)  A Z Xy  , 

-X/  ~ -V  ~ L.  ~ ~ ~ _L~ 


where 


A!  (A!  , • • • » A „) 
~i  ~il  2 

IP 


This  means 


(3.2.12) 


3 log  £(y) 
30. 


-U 


„-l 


>( i 2 © y'£  y 


Vec  E 


-1 


We  shall  use  this  latter  form  for 
derivative  of  log  £(y). 


3 log  £(y) 


when  finding  the  second 


3.3.  The  Numerical  Approximations 
The  equation 


3 log  £(y) 
30 


0 


is  clearly  nonlinear  and  cannot  be  solved  explicitly.  So  we  will  use 
numerical  approximations  to  get  asymptotically  efficient  estimates. 
The  methods  we  are  going  to  use  are  the  Newton-Raphson  method  and  the 
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^coring  method.  In  both  methods  we  need  an  initial  consistent  estimate  (of 

order  T ) of  0,  call  it  then  the  Newton-Raphson  method  consists 

of  solving  the  following  system  of  linear  equations  for  0, 

~(1) 


(3.3.1) 


3 log  £(y) 


30  36’ 


e,§  (2(d~5(o))  ‘ 

~ "(0) 


3 log  £(y) 


30 


2"2(o) 


3 log  £(y) 

The  Scoring  method  is  the  same  as  above  with  g,  — replacing 

90  90 f r & 

32  log  £(y) 

0g  gg,  j i.e.  we  have  the  equations 


(3,3.2)  -e 


'3  log  £(y) 
30  30' 


0=0 


(0) 


(®(1)  ®(0)} 


3 log  £(y) 
30  " 


10  ~ (0) 


To  find  9^  we  estimate  the  covariance  matrices  by  their  sample  ana- 
logues , so 


(3.3.3) 


>.  . T-s 

Z = J-  Y y y. 
~(0)  T-s  ^t~t+s 


s 0 j I5  • . * y q , 


3.4.  The  Scoring  Method 

To  arrive  at  the  linear  equations  for  this  method  we  need  to  find 


^32  log  £(y) 


V 


30  30’ 


but  we  know  that 


^32  log  £(y) 


&\ 


\ 


30  30’ 


= -€ 


f3  log  £(y)  3 log  £(y)\ 


30 


30' 


and  from  (3.2.11) 

fd  log  £(y)  3 log  £(y)\ 


(3.4.1)  e 


30 


30' 


.-1 


© 5 \ ~ Vec  2~X)(y'£  1 © y'2"1  - Vec'  Z 1)  ] EG 


* v ^ „ t v~l 
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Now 


z = E 1y  ~ N(0,E  1) 


and  we  need 


£(z  © z - Vec  E 1)(z'  © z'  - Vec'  E 1) 


I V 


= e(zz'  © zz')  - Vec  E 1 Vec*  E 1 


since  £(z  © z)  = Vec 


,-l 


So,  suppose  u ~ N(0,  D ),  we  want  to  find  £(uu'  © uu') 

th  ~ n*n 

ij  block  of  uu'  © uu'  is 


u . u . uu 
1 2 — 


To  find  e(Uj[u  uu')  we  use  the  result  (Anderson  (1958,  p.  39)) 


£(u.u.u  u ) = d..d  + d,  d + d d 
i 3 r s ij  rs  ir  js  is  jr 


which  yields 


where 


e(u.u.  uu')  = d..  D + d.d!  + d.d!  , 
1 3 ~~  ij  ~ ~i~J  ~J~i 


D = (d  , . . . , d ) . 
~ -i.  n 


From  this  we  can  get 


The 
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/d  i\ 


e(uu’  © uu')  = D © D + 


+ 


. 

(ds,  d') 

•vl  ~n 

d 

W 

d1 

d„d  ’ 

d d’ 

~1~1 

~2~1 

~n~l 

d.d  ’ 

d-d’ 

d d’ 

\~l~n  ~z~n 

~n~n 

\ 


Now,  the  last  matrix  was  shown  by  Magnus  and  Neudecker  (1977)  to  be  equal  to 

K,(D  © D)  , 

where 


~n 


(3.4.2) 


K = 
~n 


/E(n)  E(n) 
' ~11  ~21 


\ ^ 4n) 

\~ln  ~2n 


E(”)\ 

~nl 


E(n), 
~nn  / 


,(n) , 


and  E..  ’s  are  nxn  matrices  as  defined  before.  So 

£(uu'  © uu')  = (I  „ + K )(D  0 D)  + Vec  D Vec’  D . 

— . — ~ z ~n  ~ ~ ~ ~ 


Now  for  our  problem 


u = z,  D=z\n;=Tp 


which  means 


(3.4.3)  e(zz’  © zz')  = (I  9 0 + K^)(E  1 ® E X)  + Vec  I 1 Vec’  Z X 


-t2p2 


Finally  we  get  the  average  information  matrix  as  1/T  times 

&P-  £ • --je’  = * GE*  (I  2 2 + K )(E4  © r1)  EG’ 

~ ' Tp  V ~ 
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So  the  linear  equations  for  the  scoring  method  are 


(3-4-5>  ~,(Vp2  + v(ho>  © kip  ?=’(®<i)-!(o)) 


'2“'<5(3)?:®ho)3:-Vec5(S)>  • 


Once  we  get  we  could  use  that  as  ®(q)  i-n  (3.4.5)  and  get  a 

second  iterate  hut  for  large  samples  this  is  not  necessary. 

~(2) 

3.5.  The  Newton-Raphson  Method 
3.5.1.  Preliminaries 

To  write  down  the  linear  equations  for  this  method  we  need  the  second 

derivative  of  the  log  likelihood  function.  To  derive  the  latter  we  first 

derive  — — log  £(y)  , using  the  form  (3.2.12)  for  9 and  then 

36  30'  ~ 30 

use 


(3. 5.1.1) 


32  log  St  _ r 9 2 log  & rt 
9?  9?'  " ~ 36  30 ' ~ 


3.5.2.  The  Derivation  of 


3 log  l 
36  38' 


As  in  (3.2.12)  we  have 


9 log  £ = h(l  9 © y'E  2)  A.E  \ - h e!  Vec  E 1 , 

30.  p ~ ~ ~1~  ~ 


l 6,  I,  * • • j q 


We  have  the  derivative  of  the  second  term  w.r.t.  0 as 


(3. 5. 2.1) 


h E,(E  1 © E 1)  E , 


16 


using  (3.2.8) . It  remains  to  find  the  derivative  of  the  first  term. 

To  do  this  we  shall  find  the  derivative  w.r.t.  6 for  i _>  j and  using 
symmetry  will  complete  the  derivation.  So  we  let 

li  = Ii(®}  “ 2 ® i = 0,  1,  q 


and 


(tA.  • 


Now 


9x  . 

~ 1 


8a 


(0) 


[I  ? © y'E  X(I  © E ) E X]  A.l  \ 

~ ~ ~ i-  ^liv  ~ ~ 1 ~ -> 


uv 


2 
P 


h(I  9 © y’Z  X)  A.Z^CT  © E )Z~K  . 


~i~  ~T  ^ Zuv7: 


in 


Now  using  (A  © B)  (C  © D)  = (AC  © BD)  , we  can  factor  out  (I  © y’Z-1) 

~p 

to  the  left  and  also  can  factor  out  E y to  the  right,  which  results 

8x. 

ii  = -*<; 

p 

•vu  V 

where 


(0)  ~ ~h(l  2 © Z'l  X)  5i0,uv  5 \ > 


“ [I  ? © (IT  © E )Z  -1]  A.  + A.z  1 (I  © E ) . 

~ ^ ~T  ~uv  ~ ~ i ~uv 


'i0,uv 


-1. 


This  leads  to 

3t.  / 


92o 


3t.  3t. 

~i  ~i 


9t  . 


(0)’  a/T(0)»  •••»  3a(0) 


vsr  9a2i 

= -h(I  y © y’E"1)  (C. 


~^a  2 © y’E  X)  CiQ(I  2 © E~iy)  , 

p ~P 


10,11  ? ?’•••’  Si0,pp  l 
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where 


?iO  (~iO,ll’  -10,21’  Si0,pp)  ’ 1 °*  1»  *•**  q ‘ 


Similarly 


3x  , 


^ = -ha  9 © y’Z  X)  C.,(I  9 © E xy)  , 


-1 


30! 

~J 


ij  2 


where 


c..  (C.  . . i $ C.  . • j c . . ) 

-il  - 11,11  -11,21  -11, PP 


and 


C..  = [I  9 © (Lj  © E + L*j  E )£  1]  A.  + A.E  1(Lj  © E + 

-il, uv  ~ 2 w ~ ~uv  ~ ~vu  - ~i  ~i~  ~ w ~uv 


0,  1,  cj, 

2 , • • • p • 

symmetry  we  have 

2 © y'E"1)  C'  (I  2 © S-1y)  . 
' ~ ~ ~ J ~P 

Now,  for  j > i define 

C. . = C!  . 

-il  ~li 

and  define 

~ = (-ij)  ’ 1,J  = °,  i,  •••,  q , 

then  from  above  we  have 


j 1 i,  i 

u,v  = 1, 

Finally  for  j > i using 


3t 

— = -ha 

30!  ~{ 

~1 
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iJ  ! 


3r  / \ 

<3'5'2'2>  igT  ‘ ® Z'f  > EijCJ.j  © ?'ly) 


» j 0 , • 1 j • • * , q 


~h[lq+ 1 ® (l  2 ® y'5  X)J  £[I  © (I  2 © Z'1 

P ~p 


Now  notice  that 


I ® (I  © A)  = I © A . 
~n  ~m  v - ~mn  w ~ 


So  (3. 5. 2. 2)  becomes 


3t 


= -hd 


30'  "p^Cq+l) 
Finally  from  (3. 5. 2.1)  we  have 


© y'Z  X)  c(i  © z"1y) 

~ p (q+l)  ~ - 


(3. 5. 2. 3)  8 lQg  1 - -A 


a*  aBt  2 © y'^'1)  cd  © z-1 

88  8®  P (q+l)  ~ ~ ~p2  (q+l) 


+ h E' (Z-1  © S X)  E . 


3.5.3.  The  Equations 

From  (3. 5. 2. 3)  using  (3. 5. 1.1)  we  have 


A -i.°g  A = _L. 


36  30' 


G(I 


“p  (q+i) 


© yT1)  c (i 


2 © Z \)  G’ 

p (q+l) 


+ h GE'CZ  1 0 E 1)  EG'  . 


So  the  equations  for  the  New»ton— Raphson  method 


are 


[ G ( I 


“p  (q+D 


® Z'hJ)’  ~(0)(Jp2(q+1)  ® ?(J)Z>  - S’<5(J)  © ?(£,>  BP*] 


* - 5 ?(hly®Kl-  S‘1 


(1)  “(0)'  - ™ ';(0)Z  ^ i(0)Z  - Vec  5(0) > • 
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3.6.  Summary 


In  the  preceding  sections  we  have  derived  the  linear  equations  for 
the  estimates,  using  the  Newton-Rapnson  and  Scoring  methods.  The  two 
resulting  estimates  are  asymptotically  equivalent,  but  at  this  point  it 
is  not  clear  which  one  is  easier  to  compute.  To  write  down  the  equations 
we  first  have  to  invert  the  Tp*Tp  block  Toeplitz  matrix  E,n..  and  once 
we  have  the  equations,  we  need  to  find  the  best  method  (computationally 
easiest)  to  solve  them.  These  problems  will  be  considered  in  future 
work. 

To  compare  the  computational  problems  with  that  of  Reinsel  (1976),  we 

A 

see  that  in  the  latter  a matrix  G which  is  essentially  of  the  same  form  and 

size  as  E . . has  to  be  inverted  (G  = A.  © Li) . But  once  G ^ 

~ ^ u ) ~ i-u  ~ 1 ~ ~ 

is  computed,  Reinsel  has  shown  that  the  solutions  to  his  equations  are 
the  same  as  some  generalized  least  square  estimates. 
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4.  Estimation  In  the  Frequency  Domain 
4.1.  Introduction 


For  a stationary  process  (z^  t = 0,  + 1,  ...}  with  covariances 

?s  =£(?t^t+s)’  s = °.  ± 1.  •••,  the  spectral  density  matrix  f is 
defined  as 


(4.1.1) 


f(A) 


2tt 


t,  -isA 
D e 
~s 


The  covariances  can  be  recovered  from  f(*)  via 

rl T 


D = 
~s 


f(A)  e±sA  dA 


-TT 


The  sample  analogue  of  the  spectral  density,  the  periodogram,  is  defined 
as 


(4.1.2) 


1(A) 


2ir 


T-l 

I 


-(T-l) 


D e~isA 
~ s 


where  is  the  sample  analogue  of  Dg,  more  precisely 


D 


~s 


T-s 


I l 


t=l 


y y 

itit+s 


We  can  also  represent  1(A)  in  terms  of  the  discrete  Fourier  transforms 

(4.1.3)  1(A)  = w(A)  w*(A)  , 
where 

(4.1.4)  w(A)  = -L-  l z einA  . 

/2W  n=l  ~n 

For  a fuller  treatment  see  Anderson  (1971) . 

If  the  process  {z^  t = 0,  + 1,  ...}  is  Gaussian  then  the  log  like- 
lihood function  may  be  approximated  by 
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(4.1.5) 


-%  log  | T | -hi  tr[f  1(A  ) 1(A)] 

~ ~ u ~ L 


where 


r cov(z^5  • • » * 5 


A = t = 0,  1,  . . . , T— 1 . 


Whittle  (1953,  1961)  suggested  this  for  the  case  p=l  and  Dunsmuir  and 

Hannan  (1976)  showed  that  this  leads  to  efficient  estimates  for  general  p. 

— (s) 

For  our  problem  y=z,D  EE  , F = E,  and  we  only  have  a finite 

~ ~ ~s  ~ ~ ~ 

number  of  nonzero  covariances  so 


(4.1.6) 


f (A)  = ~ l E(s)  e iXs 
-q 


and 


i T-l 

‘w  l 5s e 

1 -(T-l)  S 


-iAs 


So  the  log  likelihood  can  be  approximated  by 
(4.1.7)  log  Z & -h  log  | S 1 -hi  tr^1  Jt)  » 


where  f = f(A  ),  I = 1(A).  We  shall  use  the  same  approximation  methods 
t ~ t ~ t 

as  in  the  time  domain  and  will  also  use 


(4.1.8) 


3 log  £ _ 3 log  £ 

~ 30 


(4.1.9) 


= o 

V 30  7 


22 


4.2.  The  Derivation  of 


Using  (4.1,7)  and  (4.1.9)  we  get 


(4.2.1) 


3 log  l 


= l ~ tr(f"1  I ) - £ — tr (f  1 I ) 

t L 36  ~t  30  ~t  . 


9f 

~ ^(f"1 1 j = -^(f"1  f"1 1 ) 

hn  ~ t ~ £ OX  ~£  ~ t 7 


Differentiating  (4.1.6)  yields 


2ir  5Uv  s 


1 -iX  s iX  s 

Trr  E + e E ] , u.v 

zn  ~uv  ~vu 


1 , • • • t p 


Jo T "(h1 


l -] 

-7TZ  tr(f  E f I ) 
2tt  ~t  ~uv  ~t  ~t 


h “(j;1  sus;  s;1  it> 


1 . ^-1  t ^-1 

2ir  2V  ~t  It  !t  ~u 


9^  (h  ) , u,v  = 1, 

Z7T  ~t  VU 


where 


(4.2.2) 


h - h1  h h1  • 
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and  we  have  used  the  fact  that  E = e e1 , where  e . e are  the  u 

~uv  ~u~v  ~v 

t h 

and  v column  of  the  pxp  identity  matrix.  Now  we  can  easily  see 

(4.2.3) 


th 


30 


^ xt>  - 


-^Vec(hp 


0 


Similarly 


3a 


b t it' 


2tt 


tr 


uv 


■ 1 -iX  s iX  s _ - 

f (e  E +etE)fiI 

~t  ~uv  ~vu  ~t  ~t 


_L_ 

2tt 


■ -iX  s iX  s ' 

e £ h*  + e h 
~t  ~t 


uv 


, , u,v  = 1 


9 ■ • “ 9 F 9 


which  yields 


(4.2.4)  ~ tr (f  1 I ) 

30 

~s 


iX  s -iX  s 

- ~2jt  Vec(e  + e t hp  , s = 1,  . ..,  q 


To  complete  the  derivation  of  ^ ^ we  have  to  take  the  expectation 

30 

of  (4.2.3)  and  (4.2.4).  This  yields 


30 


"(j;1  it)! 


o 


e 


pi  _1 

I 

t 

CO 

mi 

-IX  s , -j 

: tr(f  1 I ) 
130  ~C  11  J 

= - Vec 

2it 

e + e 

f 1 
~t 

since  £h^_  ~ f h which  follows  from  £(ip  = ffc  + 0(T  ^)  . Now  let 

£ = h - f-1  , 

~t  ~t  ~t  ’ 


then  using  (4.2.1)  we  have 
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(4.2.5) 


3 log  £ 1 v 

= * l Vec  a ■ 


(4.2.6) 


3 log  £ _ _1_ 
4 IT 


30 


f ±X  s -iA  s ■ 

l Vec  e £ + e C £' 
t L ~c  -t 


> ® 1>  . . . , q 


Now,  from  (4.1.6)  it  is  obvious  that  f'  = f or  f = f*t  where  * 
indicates  "conjugate  transpose",  i.e.  f is  Hermitian.  Also  I’  - 

which  leads  to  h-  - h and  V . 1.  Be  can  use  this  to  simplify 
(4.2.6)  as  follows 


iA  s 


3 log  £ 1 v iA  s xa  a 

= ^fIVec(e  A ) + ^ I Vec(e  £ ) 

t t ~t 


30 

~s 


iA.  s 


1 v t 

- H l Vec(e  l ) , 

t ~L 


because  the  first  sum  is  real.  The  reason  for  this  is  that 


iA  -iA  -i^rt  i2Tf  (T-t) 
s C - e = e T = e T - -T-t 


= e 


This  means  that  for  any  real  function  r)(*) 

T-l 


T-l  iA 

I 

t=0 


I n(e  C)  = n(0)  + l n(elAt)  + , 

t=0 


for  T odd,  and 


T-l 

2 iA 


T-l  iA  2 iA  iA- 

I n(e  ) = n(0)  + l n(e  c)  + n(e  fc)  + n(-i)  , 
=n  t=0 


t=0 


25 


; mi 


vT-1 

for  T even.  So  At_Q  P(e  ) is  real.  The  same  argument  allows  us 
to  rewrite  (4.2.5)  as 


^*-1  - f l Vec  £ 


Finally  we  can  conclude 


(4.2.7) 


^2S_l  = f l j vec  £ , 

2tt  f*  ~t  ~t  ’ 


where 


~(^l/>je  X/j,  • • • » e I 0) 


i 2’  t 2 
P P 


There  is  an  alternative  form  for  (4.2.7)  which  will  be  more  useful 
in  deriving  the  second  derivative  of  log  £.  It  is  obtained  by  noticing 
that  we  can  find  a matrix  M,  such  that 


Vec  £ = M Vec  V , 

~t  ~t 


which  then  enables  us  to  rewrite  (4.2.6)  as 


(4.2.8) 


3 log  £ = 1_ 
™ 4tt 


iX  s -iX  s 

l (e  I „ + e 1 M)  Vec  £ 


It  is  easily  verified  that  in  fact 


M = K , 

~P 

where  was  defined  in  (3.4.2).  So  the  alternative  form  for  the  first 


derivative  of  log  £ is 


(4.2.9) 


, f i „ Vec  . , 

s 4tt  ‘r  ~t  ~t  ’ 
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where 


iA 


-iA 


qiA  -piA 


(4.2.10)  H'  = H'(A  ) - (I  2,  e tI2+e  tK,  ...,  e tI„+e  "K) 


4.3.  The  Second  Derivative  of  log  £ 
Using  (4.2.9)  we  have 


(4.3.1) 


i . 3 Vec  £ 

d lpg  & _ _L  v u ~t 


47T  I Ht 

39  30'  t 


30’ 


Now 


£t- = C1 f / - f_1 

~t  ~t  ~t  ~t  ~t 


which  yields 


|£ . .£-i  £ f-i  t £-i . £-i , f-i  »u  f.i  + -l.  rl 

3x  ~t  3x  ~t  ~t  ~t  it  it  it  3x  it  + it  3x  ~t  * 


which  in  turn  yields 
3 Vec  £ 


(4.3.2) 


© r1)  - 


3x 


<!t_1  ® it> 


- ® £t1)]  Vec 


3 x 


* it  Vec  . 


say.  Now  for  x = we  set 

uv 


3 Vec  £ 


t 1 


3a 


(0) 


2tt  $t  Suv  ’ 


uv 


where 


euv  — Vec  E , u,v  = 1,  ... , p . 
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So 


(4.3.3) 


9 Vec  Z 


t 1 


dQJ0 


2tt  $t  (~11’  ~21  ’ 


* ’ ® ) 
~PP 


2tt  $t  i 2 2tt  $t 


Similarly 


3 Vec  Z , -iX  s iX  s 

~t  1 A , t t ^ 

7~\ — = -7^:  <J>  (e  e + e e ) , u,v  = 1,  ...,  p 

,(s)  2ir  It  ~uv  ~vu  ’ ’ ^ 


9a' 


uv 


Now 


e = Vec  E 
~vu  ~vu 


Vec  E'  = K Vec  E = K e 

~uv  ~p  ~uv  ~p  ~uv 


which  means 


3 Vec  l 


t 1 


-iXt 


9a 


(s) 


2?*t  (e 


uv 


iX  s 

s t 

I « + e K ) e , u,v  = 1,  ....  p , 
~ 2 ~p  ~uv 

P 

S 1 j . . . 9 (J  • 


Finally 


9 Vec  Z 


(4.3.4) 


t 1 


99' 

~s 


2tt  *t  (e 


-iX  s iX  s 

I 2 + e t K ) , s = 1, 
~P  ~P 


So 

(4.3.5) 


9 Vec  £ 


90' 


1 '’■^t 

2¥$t(!2’e  ^2+e  V 

P P 


■qiX  qiX 

Z I 9 + e C K ) 
P 


1 * 
-r-  <f>  H . 
2t r ~t  ~t 
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Now  we  get  the  second  derivative  from  (4.3.1) 


(4.3.6) 


3 log  Z 1 v „ . * 

— Z 2 i Hr  t 

39  30'  8n  t ~Z  ~t  ~fc 


4.4.  The  Newton-Raphson  Method 

As  in  the  time  domain  we  will  use  zj^'s  as  inlt±al  estimates  of 
the  covariances,  and  so 


f =I  J ’ UtS  g(s) 

:t(°)  2ir  JLq  5(0) 


will  be  the  initial  estimate  of  f ^ . Accordingly  we  form  and 

$t(0)* 


l _ 2-1  T 2-1  2-1 

-t(0)  ~t(0)  it  it(0)  " !t(0)  ’ 


i(0)  ■ <!tco>  ® K(0))  - ® !t(o>  Jt  JtJo)> 


1 _ 1 * _1 

- (fwix  i’  - 1 


t(0)  it  it(0)  ® it(0)) 


Using  (4.2.7)  we  have 


(4.4.1) 


_3  log  Z _ 3 log  Z 

3?  " 2 35 


5?  Z “t  Vec  !t 


Similarly  from  (4.3.6)  we  get 


(4.4.2)  Z = -_L_  V rH  A u*  p » 

30  30’  _ 2 i ™t  It  ~t  5 ' 

~ ~ 8f  t 

So  the  linear  equations  for  the  Newton-Raphson  method  are 
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(4.4.3)  (-  l GHt  $ } H*G’)  (0 


(1)  ~(0) 


> - 2 1 st  Vec  !t<o)  • 


4.5.  The  Scoring  Method 

As  mentioned  earlier,  to  get  the  linear  equations  for  this  method 
2 

we  replace  by  its  expectation  in  the  Newton-Raphson  method. 


The  latter  is 


92  log 
30  30 


jA  i r * 

— - "S'  l GH,  h.g’ 

> 8ir  t 


which  leads  to 


S(it>  - It  ■ 


£(it>  ~ e( S 1 it  it1)  - it1  • 


which  in  turn  leads  to 


£(<f>  ) ~ "(f’"1  © f/) 


* - ^2  l St 'it'1  * &&  • 


ignoring  the  terms  of  order  T . Finally  the  linear  equations  for  this 


method  are 


StSt'itW)  ® i;(0))SS'l'S(1)-8(0))  - 2 I 2t  Vec  Jt(0)  . 
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4.5.  Remarks 


As  in  the  time  domain,  there  are  computational  problems  to  be  con- 
sidered in  setting  up  and  solving  the  equations  derived  in  the  preceding 
sections.  In  this  case  the  main  problem  in  setting  up  the  equations  is 
the  inversion  of  pXp  matrices  f ^ , t = 1,  . . . , T-l.  And  again  we 
have  to  find  the  best  way  to  solve  the  resulting  equations.  It  seems 
that  the  computation  of  the  estimates  in  the  time  domain  is  easier  than 
that  in  the  frequency  domain. 

Comparing  with  Nicholls  (1976)  and  Anderson  (1978)  which  deal  with 

the  estimation  of  the  coefficients  A , i = 1 q,  we  see  that  the 

main  problem  of  inversion  of  f (0)'s  is  also  Present  in  these  papers. 
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5.  Asymptotic  Properties 


The  four  estimates  proposed  in  this  paper  are  asymptotically  equi- 
valent and  we  shall  show  that  they  are  efficient,  i.e. 


Sf  (§(1)-6)  + N(0,  «9  1(0))  , 

£ 

where  dl(0)  is  the  limiting  average  informtion  matrix  and  indicates 


convergence  in  distribution. 

To  find  J(0) , by  definition  we  have 

(5.1)  J(8)  - 

£ * 2X'1  ® S1)*' • 


l 

16tt3 


2tt 


GH(f  1 0 f 1)h’<G'  dA  , 


where  the  argument  A is  omitted  from  H and  f. 

The  four  estimates  are  obtained  from  equations  like 


(5.2) 


*!(0)>  <!(i)-§(0)> 


0=0 


(0) 


where  -^(0/n\)  is  a consistent  estimate  of  d>(B).  We  can  rewrite  (5.2) 


as 


(5.3)  3(®,m)  ft  (e„,-e)  - j(8,„o  ft  (§,„,-e)  + -t  3 \°l  * 


:(oy  v~(i)  ~v~(0) 

where  0 is  the  true  parameter.  Now 


(0) 


/ f 


90 


~ ~(0) 


(5.4) 


1 3 log  t _ _1_  3 log  i 


Sf 


30 


Sr 


30 


_1_  3 log  l 


5-5(o)  ^ 


30  30’ 


0=0 


+ (5-5(o)> 
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where  1 0—0  I _<  [6-6^^  j.  Now  (5.3)  can  be  rewritten  using  (5.4) 


<5'5)  3(8(0))  /t  ce(1)-e,  - [3(*(0))  + f Vf1  0.6+ 


^ <® — -«)  +-MSM. 


~(0)  ~ 


Now  noticing  that 


1 3 log  l 
T 36  30' 


i -c9(0) 


and  ^(§(0)“®)  is  bounded  in  probability,  we  see  that  (5.5)  is  (asymp- 
totically) equivalent  to 


(5.6) 


(0Mx-0)  - cS  1(0)  — -9— 1-C?-g-£ 


Theorem.  Under  (2.1)  and  Assumptions  1 and  2 of  Section  2, 


& (0(D -0)  + N(0,  J 1(0))  , 


where  0 is  any  one  of  the  four  estimates  derived  in  this 
Proof.  Using  (5.6),  it  suffices  to  show 


paper. 


-l  us&  t : N(0 

30  ' * 


Vec(A. , . . . , A ) 
~1  ~q 


K = dg  V 


Vec  V 
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where  A.'s  and  V were  defined  in  Section  2.  Now 

~ 1 ~ 


30. 


which  means 


3 log  i _ V 3 log  l J 

- j 3®j  ‘ aq  ’ 


Q f 

3 log  SL  3 log  H 

3£  ~ 3 E,  36 


36' 

It  follows  from  Assumption  2 of  Section  2 that  is  nonsingular, 


which  means 


(5.7) 


1 3 log  l |3?'\  1 1 9 log  it 

v^r  33  ' 35  /r  3! 


But  it  has  been  shown  by  Nicholls  (1976)  and  Reinsel  (1976)  that 


£ 


^ ttO  - N(0,  *)  , 


which  is  the  same  as 


So  (5.7)  gives  us 


(5.8) 


-i..  3 log  i j . .. 

/F  99  > 


The  limiting  covariance  matrix  in  (5.8)  is  obviously  cS(9) , so 
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